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Abstract 

This letter establishes the asymptotic optimality of equal power allocation for measurements of 
a continuous wide-sense stationary (WSS) random process with a square-integrable autocorrelation 
function when linear estimation is used on equally-spaced measurements with periodicity meeting the 
Nyquist criterion and with the variance of the noise on any sample inversely proportional to the power 
expended by the user to obtain that measurement. 

I. Introduction 

Linear estimation of a random process from a set of noisy measurements is employed in many 
wireless communication and sensing applications. We consider a scenario where a continuous 
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wide-sense stationary (WSS) random process with a square-integrable autocorrelation function 
(ACF) is to be estimated using a Wiener filter from measurements that are equally- spaced with 
periodicity meeting the Nyquist criterion and subject to additive Gaussian white noise (AWGN) 
with variance inversely proportional to the power expended by the observer to obtain it. A Wiener 
filter estimator is optimal in the mean squared error sense if the random process is also Gaussian. 
Generally, if the observation period is finite in this setting, equal power allocation is suboptimal. 
However, equal power allocation is simple to implement and performs well when the observation 
period is long enough to reduce the importance of the "edge effects". In this letter we confirm 
this intuition by proving that the optimal power allocation across equally-spaced measurements 
indeed tends to equality in the asymptotic case of an infinite number of measurements of a WSS 
random process with a square-integrable ACF. 

Our scenario frequently arises in pilot symbol assisted modulation (PSAM) on wireless chan- 
nels. A Gaussian random process with a square-integrable ACF often govems the behavior of 
the wireless channel, ensuring the optimality of Wiener filtering of the channel measurements 
collected by the receiver from the known pilot signals inserted into the transmission by the 
sender at the Nyquist rate for the process. Many PSAM techniques correspond to different 
wireless channel models (see survey dD and references therein). However, we prove a general 
result, confirming the intuition behind equally- spaced equal-power pilots used in the original 
work analyzing PSAM flU (and similar approaches). 

In the following section we formally state our problem. In Section UlI] we establish preliminary 
results that facilitate the proof of the asymptotic optimality of equal power allocation for linear 
estimation of WSS random processes in Section |IV] Section |V] concludes the letter. 

II. Power Allocation Problem 

Consider a continuous-time WSS random process x{t) with a square integrable ACF Rx{t): 
jToo \Rx{t)\'^ d'T < oo. We sample x{t) at rate ^ that meets the Nyquist criterion, collecting n 
samples of x{t) from the interval [0, {n — 1)Ts]. Let x^-"'' = xiiTg). Typically (e.g. for PSAM) the 



discrete observations of x^-"'' take the following form: y'f''' = y p/"''xj-"'' +5^-"'', where {z'f'''} is an 
independent and identically distributed (i.i.d.) AWGN sequence with 5^^"^ ~ A^(0, cr^), and P/"-* > 
is the power used by the observer for the i"^ observation. Normalizing the observations by 
^p^(") j simplifies the analysis without affecting performance, yielding ?/-"^ = y-"^^ ^ P/"-* = 
x'f^ + z'f^ , where {^f"^ } is an i.i.d. sequence with z^"'^ ~ A/'(0, cr^/ Pi). The observer is subject to 
the peak power constraint Pmax such that P/"^ < Pmax for all i, where Pmax is a finite constant. 
Let Pr(n) = ^"J^J P/"^ denote the total power allocated to n observations of the process. 
We estimate x[^'' from ?/•"'' using a Wiener filter over the finite-time horizon [0, {n — l)Ts] 

(n) 

for an increasing number of observations n. Denote the sequence of covariance matrices of x\ 
as {R„}, where (Rn)ij = Rx{\i — i\Ts)- Since Rx{t) is square integrable and since x{t) is 
sampled at the Nyquist rate, the sequence {Rx{kTs)}^Z^ that forms R„ is square summablqlj: 
Y^^=-oo \Rx{kTs)\'^ < oo. Since the noise z^^^ is i.i.d., the sequence of covariance matrices of the 

( 2 2 2 \ 

. . . , defines a sequence 

of diagonal matrices. Note that the observation process depends on the sequence of power 
allocation vectors {p(")} through D„, where p(") = [Po^"\ Pi^"\ . . . ,Pi!.\]. By [ffl Eq. (12.53)], 
x(") = R„ (R„ + D„)~^ y(") defines the sequence of estimate vectors {x*^"-'}, where the 
row of matrix R„ (R„ + D„)^^ contains the Wiener filter coefficients for the estimate of x'f'\ 
Denoting the expectation operator by E [■] and the transpose of matrix A by A^, the covariance 
matrix M„ = E e(") (e^"))^ of the estimate error e(") = x(") - x^") is [31 Eq. (12.55)]: 

M„ = Rn — R„(R„ + D„)~^R„ (1) 

= (R-' + B-Y' (2) 

where ^ is due to flS] Ch. 0.7.4]. Again, note that the sequence {M„} depends on the sequence 
of power allocation vectors {p*^"-*} via D„. 

'since Rx{t) is square integrable and band-limited to it is in a Hilbert space Ti. A complete orthonormal basis for such 
is {sinc(r/r, - fc)}^=_^ [3, Ch. 3.18.3]. By Parseval's identity (H Th. 3.5(3)], /^^ \RAr)\^dr = Er=-oo \R.{kTs)\^- 
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We are interested in the relationship between the power allocation vector p^") and the mean 
squared error (MSE) of the estimate over the entire observation window, as the size of the win- 
dow, n, grows large. Note that the diagonal entries of M„ contain the MSE of each observation. 
Thus, denoting the trace of the matrix A by Tr[A], the MSE over all observations is: 

n-l 

^(P^"^)--E^[^']=-Tr[M„] (3) 
n ^ — ^ n 

1=0 

Our main result is the following theorem: 

Theorem 1. The MSE £ ^pip; j of the optimal power allocation Y>opt converges to the MSE 
£ {v^Iq^ of the equal power allocation ^^eq = {Pi"^^ '■ Pi^^ = Peq} as n ^ oo, with P^q = 

In a typical scenario when Pxin) increases linearly with n, Pgq is a constant. In order to prove 
this theorem in Section |lVl we provide several essential lemmas in the next section. 

III. Prerequisites 

We first prove that (|3]l is strictly convex over all choices of power allocation, demonstrating the 
uniqueness of the optimal power allocation. We then introduce cyclically-symmetric functions, 
and prove that, if such functions are strictly convex on a convex domain, then the unique vector 
that attains their minimum has equal values. The section concludes with the introduction of the 
asymptotic equivalence of Toeplitz and circulant matrices using material from [|6l and flTJ. 

A. Power Allocation Vector that Minimizes MSE is Unique 

Lemma 1. If A is a real symmetric positive-definite n xn matrix, then the function /(x) = 
Tr[(A + diag(x))~^] is strictly convex within the polytope Yll^=o •'^i = C, Xi > 0. 

Proof: Since the domain of /(x) is convex fW, Ch. 2.1.2 and 2.1.4], /(x) is strictly convex 
in X if and only if g{t) = /(x + tv) is strictly convex in t for any t G M and v G such 
that X + tv is in the domain of /(x) (i.e. x + tv is a real vector with positive entries that sum 
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to C) [i8j Ch. 3.1.1]. This follows directly from the definition of convexity and is known as the 
method of restriction to a line. Define B = A + diag(x) +t diag(v) and note that it is symmetric 
positive-definite (as is B^^) since A is symmetric positive-definite and diag(x) -l-tdiag(v) is a 
diagonal matrix with positive entries on the diagonal. Consider h{t) = u^B~^u, where u is an 
arbitrary non-zero vector. Its first two derivatives with respect to t are [0 Ch. D.2.1]: 

h'{t) = -u^B-Miag(v)B~iu (4) 
h"{t) = 2w^B-^w (5) 

where w = diag(v)B~^u. We can substitute w into © since B is symmetric. Also, since B^^ 
is positive-definite, h"{t) > 0, implying that h(t) is strictly convex in t. Now 

git) = Tr[(A + diag(x)+tdiag(v))-i] (6) 

n-l 

= ^enA + diag(x)+tdiag(v))-ie, (7) 

where ej is a vector containing one in the location and zeros everywhere else. Since each 
summand of (|7]) can be written down as h{t) (with replacing u) and since the sum preserves 
convexity, g{t) is strictly convex in t. Therefore, /(x) is strictly convex in x. ■ 
Since R„ is symmetric positive-definite, so is its inverse. Also D^^ = '^"^s^p' '\ Thus, by 
Lemma [U £^(p*^")) is strictly convex and p^pt that minimizes MSB in our problem is unique. 

B. Cyclically-symmetric Functions 

We next introduce a class of symmetric functions and prove a useful property about them. 

Definition 1 (Cyclically-symmetric function). /(xo,a:i, . . . is cyclically-symmetric if 

f{Xo, Xi,..., Xn-l) = f{Xi, . . . , Xn-1, Xq) 

Lemma 2. Suppose f{xQ, xi, . . . , is strictly convex and cyclically-symmetric on a convex 

domain S. If vector x* = argmin^g^ f{xo, Xi, . . . , Xn-i), then x^ = xl = . . . = 
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Proof: Since /(xq, xi, . . . , Xn-i) is strictly convex, x* is unique. Since /(xq, xi, . . . , x„-_i) is 
cyclically-symmetric, then, for alH = 1, . . . , n—1, x* also minimizes /(x,, . . . , Xn_i, xi, . . . , Xj-i) 
Thus, ^ — — . . . — — ^Yi—X' ^ 

C. Asymptotically Equivalent Matrices 

Results on the asymptotic equivalence of matrix sequences in [6, Ch. 2] enable the discussion 
of the Toeplitz and circulant matrices at the end of this section. First, let A be a real-valued 
n X n matrix. Then we define the matrix norms as follows: 



Definition 2 (Strong norm). ||A|| = max^-zTz^i [z^A^Az] 



1/2 



Definition 3 (Weak norm). | A| = Tr [A^A]. 
If A is symmetric positive-definite with eigenvalues {\i}^~Q, |A| = \/^ J2i=o ^i- Also, ||A 



Amax and ||A ^|| = l/Xmm, where Amax and Amin are the maximum and minimum eigenvalues 
of A, respectively. 

Lemma 3 (Lemma 2.3 in [[6l). For n x n matrices A and B, |AB| < ||A|| ■ |B|. 

Now define the asymptotic equivalence of matrix sequences as f6^, Ch. 2.3]: 

Definition 4 (Asymptotically Equivalent Sequences of Matrices). The sequences ofnxn matrices 
{A„} and {B„} are said to be asymptotically equivalent if the following hold: 

||A,||,||B„|| <M< oo,n = 1,2,... (8) 
lim„_,oo |A„ - B„| = (9) 

We abbreviate the asymptotic equivalence of the sequences {A„} and {B„} by A„ ~ B„. 
Properties of asymptotic equivalence are stated and proved in [6, Theorem 2.1]. A property 
particularly useful in the proof of Theorem [T| is re-stated here as a lemma: 



Lemma 4. A„ ~ B„ and ||A„^||, ||B„^|| < < oo, n = 1, 2, . . then ~ B„\ 
Proof: \A-^ - B;i| = |B;^iB„A;i - B^^A^A^^j < ||B„i|| • ||A„i|| ■ |A,„ - B,„| - 

'IL — ?-CX 

where the inequality is due to Lemma |3l 

Another important consequence of asymptotic equivalence follows from [6, Corollary 2.1]: 



Lemma 5. If An ~ B„, then lim^^oo ^ Tr[A„] = lim„_j.oo ^ Tr[B„] when either limit exists. 



Proof sketch: By the Cauchy-Schwarz inequality, 



TrfA„-B„, 



n 



< lA. 



B. 



0. 



D. Sequences of Toeplitz and Circulant Matrices 

An n X n Toeplitz matrix T„, illustrated in Fig. |l(a)| is defined by a sequence {t^""*} where 



tj_j. The covariance matrix R„ in Section HI] is Toeplitz and symmetric. An 



n X n 



circulant matrix C„, illustrated in Fig. |l(b)| is defined by a sequence {4"^} where (C„). 



C(jlj) mod n- Sincc the sequence {Rx{kTs)}'^^Q that defines R„ is square summable, we can define 
an asymptotically equivalent sequence of circulant matrices C„ ~ R„ using [7, Eq. (7)]: 



k 



n 



The resulting circulant matrix C„ is symmetric since, by (|TOl ). c 



C. 



(n) 
n—k' 



(10) 



An) An) 



An) 



-(,1-2) 



-f(n) -f(n) 

■-n-l ''n-2 ' ' ' "-O 



(a) Toeplitz matrix 



c. 



„(") „{") 

(") J") 



Ci (_.o 



„(") 



(b) Circulant matrix 



Fig. 1. Illustration of Toeplitz and circulant matrices. 



By E Eq. (5)], ^ F^^A^F^, where A„ = diag i z/, 



entries z/^!'"^ = (F„R„F„^)^^ of the covariance matrix of the discrete Fourier transform (DFT) 



,(n) 



n-l 



i=0 



contains the diagonal 



8 



F„x*^") of x*^") and (F„)j jt = -XeSTnfcj/n ^^iq DFT rotation matrix. Since R„ is positive-definite, 
by the properties of the similarity transformation, F„R„F^^ is positive-definite and has positive 
diagonal entries. Thus, A„ is positive-definite and so is C„. 

IV. Asymptotic Optimality of Equal Power Allocation 

Proof (Theorem [7]).- Define symmetric circulant matrix C„ as in Section IIII-DI so that 
Cn ~ Rn and consider Q. Asymptotic equivalence results in the following chain of implications: 

C„~R„ ^ C-'r^R-' (11) 

C-' + B-'^R-' + B-' (12) 

^ {c-' + B-T'-{R-' + rf-T' (13) 

lim -Tr[LJ = lim - TrfMJ (14) 

where L„ = (C,^^ + D^^) ^ . Since R„ and C„ are symmetric positive-definite, the conditions 
for Lemma |4] hold, resulting in (fTTT ). Then (fT2] ) follows from adding D^^ to both sides of the 
asymptotic equivalence relation and noting that the condition ([8]) is satisfied via Weyl's inequality 
im Theorem 4.3.1] since the peak power constraint on the observer implies that D^^ has finite 
eigenvalues. Lemma |4] yields (fT3l) . and (fT4l) is due to Lemma [51 Let 

^equiv(p(")) = -Tr[(C;^ + D;i)-i] (15) 
n 

Then, since we defined £^(p'^")) = ^Tr[M„] in ([3]), (fT4l) can be restated as follows: 

lim^(p(")) = lim fequw(p^")) (16) 

Since C„ is symmetric positive-definite, by Lemma [U i^equiv(p '■"'*) is strictly convex in p*^"). 
Showing that £^equiv(p*^"'') is cyclically-symmetric with respect to p*^") would complete the proof 
by Lemma |2l The discussion of the convergence of £(p(")) to £:equiv(p^"^) follows the proof. 
Denote the similarity transformation 7^i(A) = SjASj"^ of an n x n matrix A the rotation 

\(n—i)x{n—{) 



of degree i, where Sj 



!?■ X i 



and Inxn is an n X n identity matrix. Sj is 
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a permutation matrix, and is thus orthogonal, implying that S^'^ = Sf = S„_i. Suppose that 
the rows and columns of matrix A are labeled 0, . . . , n — 1 top-to-bottom and right-to-left, 
respectively. Then Sj A produces a matrix with the top i rows of A shifted to the bottom (i.e. rows 
0, . . . ,i — 1 become rows n — i — 1, . . . ,n — 1), and AS^^ produces a matrix with the left i 
columns of A shifted to the right (i.e. columns 0, . . . — l become columns n — i — l,...,n — l). 
Shifting the top i rows of a circulant matrix C down produces the same matrix as shifting the 
left n — i columns to the right. Thus, SjC = CSj, which implies the rotation invariance of 
circulant matrices: 7^i(C) = C. 

Inverse A"^ of matrix A can be expressed as (A^^)-^. = ^'^^(^M.jA-^) where det(A) 
denotes the determinant of A and A^,; j (A) = det(A(*'-'^) with the sub-matrix A'^*'-'^ formed by 
removing row i and column j from A |l5l Ch. 0.8.1]. Thus, (fT5l) can be re-stated as: 

c — X]fc=o -^fc.fclC^^ + D„^) 

'^"""^P ' ~ ndet(C-i + D-i) ^ ^ 

The inverse of a circulant matrix, if it exists, is circulanlB [6, Theorem 3.1 (3)]. Due to 
the rotational invariance of circulant matrices, for all i = 1, . . . ,n — 1, 7^j(C^^ + D^^) = 

p(>i) p(") p(n) p('i) 

+ MB-') with 7^,(D;l) = diag(^, . . . , . . . , ^). The denominator of ^ 

is cyclically-symmetric with respect to p*^"^ due to the rotation being a similarity transformation, 
which preserves the determinant. Since the submatrix of C^^ + D^^ with row k and column k 
removed is a submatrix of C^^ + 7?.i(D~^) with row {k — i) mod n and column {k — i) mod n 
removed, the numerator of (fTTl i is also cyclically-symmetric with respect to p*^"-'. Therefore, 
^equiv(p^"'') is cyclically- symmetric and, by Lemma |2l argminp(„) £equiv(p*-"^) = piq^- By (fT6l) . 

i^equiv(Peq^), Completing the proof. ■ 
Convergence: First bound |^(p^"^) - ^equiv(p^"^) | = |^Tr[L„ - M„]| < |L„ - M„| using the 
Lemma [5] proof idea. Now, as done in the proof of Lemma HI apply Lemma [3] |L„ — M„| < 
||L„|| ■ ||M„|| ■ ||C^^|| ■ ||R^^|| • |C„ — R„|. The strong norm terms are bounded by Weyl's 



^We note that, in general, the inverses of Toephtz matrices are not Toeplitz. 
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theorem per the arguments following (fT4T i. while the discussion following [7, Eq. (10)] asserts 
that |C„ — R„| < + e where C and e are constants, with e arbitrarily small. Thus, in ([T6] ). 
^(p*^"^) and ^equiv(p*^'^'') converge at a rate proportional to 

Finally, while the numerical results are not in the scope of this letter, our evaluations show 
that -6^(piq'') converges to i^(pipt) fairly quickljtl in typical wireless communication scenarios. 

V. Conclusion 

The asymptotic optimality is established for the equal power allocation between equally-spaced 
measurements used for Wiener filter estimation of a continuous WSS random process with a 
square-integrable ACF, where the periodicity of the measurements meets the Nyquist criterion 
and the measurements are subject to AWGN with variance inversely proportional to the power 
expended by the observer. 



References 

[1] L. Tong, B. Sadler, and M. Dong, "Pilot-assisted wireless transmissions: general model, design criteria, and signal 

processing," IEEE Signal Processing Magazine, vol. 21, no. 6, pp. 12-25, Nov. 2004. 
[2] J. K. Cavers, "An analysis of pilot symbol assisted modulation for rayleigh fading channels [mobile radio]," IEEE 

Transactions on Vehicular Technology, vol. 40, no. 4, pp. 686-693, Nov. 1991. 
[3] T. K. Moon and W. C. Stirling, Mathematical Methods and Algorithms for Signal Processing. Upper Saddle River, NJ: 

Prentice Hall, 2000. 

[4] S. M. Kay, Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory, 1st ed. Upper Saddle River, NJ: 
Prentice Hall, 1993. 

[5] R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge University Press, 1985. 

[6] R. M. Gray, "Toeplitz and circulant matrices: A review," Foundations and Trends in Communications and Information 
Theory, vol. 2, no. 3, pp. 155-239, 2006. [Online]. Available: |http://www-ee.stanford.edu/~gray/toeplitz.pdf| 

'E.g.: We used MATLAB R2011a fmincon at default settings with Pt(?i) — n and — 1, initialized by Pcq'', to find 

f (Pop, ) for Rx{t) — e~''^', where x{t) is sampled at the rate — Hz (capturing 99% of the spectrum). This converged 

(i.e. fmincon was not able to improve f (Popt') below the initial £^(pi"-') within the default tolerance 1 x 10~®) at n = 84. 
For Jakes' model with Rx{t) — J(){2nfoT) sampled at the rate 2fo Hz (Nyquist), convergence was immediate (at n = 1). 



11 



[7] J. Pearl, "On coding and filtering stationary signals by discrete fourier transforms," IEEE Transactions on Information 

Theory, vol. 19, no. 2, pp. 229-232, Mar. 1973. 
[8] S. Boyd and L. Vandenberghe, Convex Optimization. New York, NY, USA: Cambridge University Press, 2004. 
[9] J. Dattorro, Convex Optimization & Euclidean Distance Geometry. Palo Alto, CA: Meeboo Publishing USA, 2005. 



